Technical  Research  Note  176 

AN  EXPERIMENTAL  STUDY  OF  TWO  METHODS 
OF  INDEXING  TACTICAL  IMAGERY 

by  RICHARD  S.  LAYMON. 

System  Development  Corporation 


,  “V 

JULY  1966 

Distribution  of  this  document  is  unlimited. 

BEST  AVAILABLE  COPY 

U.  S.  ARMY  PERSONNEL  RESEARCH  OFFICE 

e30  0^0^  10  ISO 


i .  s.  a  inn  i»i:hsonm;l  kkskaiuii  on  h  i; 


An  arti\  i t \  of  the  ( lliief. 


Re>eiir<*li  ami 


l)e\elo|>ineiit 


J.  E.  UHLANER  '  M.  0.  BECKER 

Director  Colonel,  GS 

USAPRO  Laboratories  Commanding 


Research  accompl i shed  under  contract  to  the  Department  of  the  Army. 

i 

system  Development  corporation 

Advanced  Systems  Division 


NOTICES 


DDC  AVAILABIL  ITY;  Qualified  requestors  may  obtain  copies  of  this  ieport  directly  from 
DDC.  Available,  for  sole  to  the  public,  from  the  Clearinghouse  for  Federal  Scientific  and 
Technical  Information,  Depoitment  of  Commerce,  Springfield,  Virginia  22151. 


DISTRIBUTION:  Primary  distribution  of  this  report  has  been  made  by  APRO  Please 
address  correspondence  concerning  distribution  of  reports  to:  U.  S.  Army  Personnel 
Reseotjh  Office,  Washington,  D.  C.  20315.  Copies  ..ioy  also  be  obtained  on  loon  from 
local  depository  libraries.  A  list  of  these  libraries  may  be  obtained  from;  Documents 
Expediting  Project,  Library  of  Congress,  Washington,  D  C.  20540 


FINAL  DISPOSITION:  This  report  may  be  destroyed  when  <t  is  no  longer  needed.  Please  do 
not  return  it  to  the  U.  S.  Army  Personnel  Research  Off'Ce. 

NOTE:  The  fmd.n  gs  m  this  report  are  not  to  be  construed  as  on  official  Deportment  of  the 
A  ,ny  position,  unless  su  designated  by  other  outhofi/ed  documents. 


FOREWORD 


The  SURVEILLANCE  SYSTEMS  RDT&E  PROJECT  has  as  its  objective  the  production  of 
scientific  reseorch  data  bearing  on  the  extraction  of  information  from  imagery  produced  as  a  result 
of  aerial  reconnaissance  and  the  efficient  storage,  retrieval,  and  transmission  of  information  within 
an  advanced  computerized  image  interpretation  facility.  Emphasis  is  on  the  men  within  the  facility 
and  their  interactions  with  equipment  in  performing  their  functions.  Research  results  are  used  in 
future  systems  design  and  in  the  development  of  enhanced  techniques  and  procedures  fcr  oil  phases 
of  the  interpretation  process  within  the  data  reduction  facility. 

The  MAN-COMPUTER  FUNCTIONS  Task  is  one  of  four  research  Tasks  established  to 
focus  on  operationally  meaningful  segments  of  the  surveillance  system.  Among  the  specific  objec¬ 
tives  of  the  Task  is  the  development  of  an  integrated  system  for  the  organization  and  presentation 
of  reference  information.  The  present  study  compares  two  methods  of  indexing  reference  materials 
for  the  use  of  image  interpreters  in  an  integrated  image  interpretation  facility. 

The  experimentation  reported  was  conducted  jointly  by  personnel  of  the  Advanced  Systems 
Division,  System  Development  Corporation,  and  personnel  of  the  U.  S.  Army  Personnel  Research 
Office.  Mr.  Arthur  Nelson,  Federal  Systems  Division,  IBM,  who  participated  in  a  related  study  cor 
ducted  jointly  with  U.  S.  APRO  research  personnel,  gave  substantial  assistance  in  developing  the 
concept  of  the  study  and  interpretation  of  the  results. 
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AN  EXPERIMENTAL  STUDY  OF  TWO  METHODS  OF  INDEXING  TACTICAL  IMAGERY 


BRIEF 


Requirement: 

To  evaluate  two  indexing  methods--graphic  and  textuai--as  aids  to  retrieval  of  stored  roils 
of  tactical  imagery  in  a  tactical  image  interpretation  facility.  The  study  was  designed  to  contri¬ 
bute  to  the  broad  elective  of  developing  maximally  effective  procedures  for  organizing  and  pre¬ 
senting  reference  materials  in  a  computer-based  image  interpretation  system. 


Pr  ocedure: 

Image  interpreters  were  trained  in  the  use  of  area-oriented  graphic  and  textual  indexes  to 
tactical  imagery  on  roll  film.  Experiments  were  then  conducted  in  which  the  interprets  were  re¬ 
quired  to  obtain  from  indexes  »he  accession  numbers  of  rolls  of  imagery  containing  coverage  for  a 
given  arec.  Interpreter  performance  with  g.aphic  and  textual  indexes  was  compared.  The  problems 
were  designed  to  require  use  of  the  indexes  in  a  variety  of  experimental  conditions  including 
variation  in  the  size  of  an  area  and  inclusion  of  an  area  in  one  map  or  in  two  mops.  Order  of  pre¬ 
senting  the  indexes  and  the  problem  conditions  were  balanced.  Time  taken  to  reach  a  correct 
solution  was  recorded  to  the  nearest  second. 


Findings: 

With  the  textual  index,  interpreters  took  less  time  on  the  average  when  the  indexed  areas 
were  small.  With  the  graphic  index,  they  took  less  time  when  the  areas  were  large.  The  longer 
time  taken  to  find  the  desired  imagery  for  large  areas  with  the.  textual  index  appeared  due  to  the 
recording  and  analysis  tasks  required  when  using  the  textual  index  under  those  conditions. 


Utilization  of  Findings: 

A  graphic  index  would  be  useful  in  a  manual  image  processing  system  where  ’  major  re 
quirement  is  to  obtain  image  coverage  for  large  geographical  areas. 

A  textual  index  which  utilizes  a  fine  grid  to  quantify  orea  coveroge  would  be  useful  in  a 
computerized  tactical  image  processing  system  where  the  major  requirement  ir  to  obtain  image 
coverage  of  targets  confined  to  small  areas. 

Where  reference  coverage  is  needed  for  both  small  and  large  areas,  a  textual  index  m:ght 
serve  as  a  broad  screen  to  isolate  sets  of  prior  coverage  imagery,  which  could  then  be  more 
thoroughly  searched  (by  graphic  index)  for  the  desired  detailed  coveroge. 
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BRIEF 


Requ  irement: 

To  evaluate  two  indexing  methods--graphic  and  textual-as  aids  to  retrieval  of  stored  rolls 
of  tactical  imagery  in  a  tactical  image  interpretation  focility.  The  study  was  designed  to  contri¬ 
bute  to  the  broad  objective  of  developing  maximally  effective  procedures  for  organizing  and  pre¬ 
senting  reference  materials  in  a  computer-based  image  interpretation  system. 


Procedure: 

Image  inierpreters  were  trained  in  the  use  of  area-oriented  graphic  and  textual  indexes  to 
tactical  imagery  on  roll  film.  Experiments  were  then  conducted  in  which  the  interpreters  were  re¬ 
quired  to  obtain  from  indexes  the  accession  numbers  of  rolls  of  imagery  containing  coverage  for  a 
given  area.  Interpreter  performance  with  graphic  and  textual  indexes  was  compared.  The  problems 
were  designed  to  require  use  of  the  indexes  in  a  variety  of  experimental  conditions  including 
variation  in  the  size  of  on  area  and  inclusion  of  an  area  in  one  map  or  in  two  maps.  Order  of  pre¬ 
senting  the  indexes  and  the  problem  conditions  were  balanced.  Time  taken  to  reach  a  correct 
solution  was  recorded  to  the  nearest  second. 


Findings: 

With  the  textual  index,  interpreters  took  less  time  on  the  overage  when  the  indexed  areos 
were  small.  With  the  graphic  index,  they  took  less  time  when  the  areas  were  large.  The  longer 
time  taken  to  find  the  desired  magery  for  large  areas  with  the  textual  index  appeared  due  to  the 
recording  and  analysis  tasks  required  wher,  using  the  textual  index  under  those  conditions. 


Utilization  of  Findings: 

A  graphic  index  would  be  useful  in  a  manual  image  processing  system  where  a  major  re¬ 
quirement  is  to  obtain  image  coverage  for  lorg ;  geographical  areas. 

A  textual  index  which  utilizes  o  fine  grid  to  quantify  area  coverage  would  be  useful  in  a 
computerized  tactical  image  processing  system  where  the  major  requirement  is  to  obtain  image 
coverage  of  targets  confined  to  small  areos. 

Where  reference  coverage  is  needed  for  both  small  and  large  areas,  a  textual  index  might 
serve  as  a  broad  screei  to  isolate  sets  of  prior  coverage  imagery,  which  could  then  be  more 
thoroughly  searched  (by  graphic  index)  for  the  desired  detailed  coverage. 
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AN  EXPERIMENTAL  STUDY  OF  TWO  METHODS  OF  INDEXING  TACTICAL  IMAGERY 


BACKGROUND 


A  Tactical  Image  Interpretation  Facility  (TIIF)  must  incorporate 
stored  reference  materials  to  assist  the  image  interpreter  in  interpre¬ 
ting  now  imagery..  These  stored  reference  materials  include  maps,  tacti¬ 
cal  target  illustrations,  keys,  image  interpreter  reports,  and  rolls  of 
tactical  imagery  containing  prior  cover.  Thu  se  materials  must  be  organ¬ 
ized  so  that  the  interpreter  can  easily  and  quickly  determine  the  avail¬ 
ability  and  location  of  specific  reference  items. 

An  extremely  important  factor  in  the  organization  of  reference 
material  is  the  indexing  system  introduced  to  represent  in  a  concise 
manner  the  contents  and  location  of  stored  material.  A  concept  for  in¬ 
dexing  reference  materials  within  a  TIIF  was  developed  in  a  related  study 
effort.1-'  The  results  of  this  analysis  are  briefly  summarized: 

The  TIIF  index  should  be  designed  to  provide  an  image  interpreter 
means  of  locating  a  specific  reference  item  by  source  data  (cate, 
sensor  identification)  or  by  selected  content  information.  Each 
type  of  reference  material  should  have  its  own  index.  For  mate¬ 
rials  indexed  at  a  central  facility  (maps,  keys,  tactical  target 
illustrations),  existing  indexes  should  be  adapted  or  modified 
for  a  TIIF.  For  reference  materials  that  are  generated  within  a 
TIIF  (prior  imagery  cover,  interpreter  reports),  the  index  infor¬ 
mation  should  be  generated  entirely  within  a  TIIF. 

Analysis  of  the  requirements  of  an  index  for  locating  and  re¬ 
trieving  desired  rolls  of  prior  imagery  indicated  that  retrieval 
of  imagery  would  be  facilitated  if  the  interpreter  searched  for 
coverage  on  the  basis  of  area  rather  than  source  data.  Area 
was  therefore  chosen  as  the  primary  basis  for  indexing. 


^Arthur  Nelson,  Kenneth  McClure,  and  John  Pol  green  (IBM),  and  Robert 
Sadacca  (U3AFR0) .  Organization  and  Presentation  of  Image  Interpre¬ 
tation  Reference  and  Auxiliary  Information.  U.  3.  Array  Personnel 
Research  Office  Technical  Research  Note  173.  June  1906.  Research  per¬ 
formed  under  contract  to  the  Department  of  the  Army  by  the  Federal 
Systems  Division,  International  Business  Machines  Corporation. 


The  TIIF  area  index  for  imagery  specifies  a  vorld-vide  grid 
containing  uniformly  equal  search  areas .  Initially,  a  one- 
degree  square  area  was  proposed  as  the  unit  search  area 
since  the  one-degree  area  is  regularly  used  by  several  mili¬ 
tary  agencies.  However,  a  study  of  TIIF  imagery  handling  re¬ 
quirements  indicated  that  a  one-degree  square  was  too  large. 

For  this  reason,  the  area  of  the  Army  Map  Service  (AMS) 

1:50,000  scale  map  (approximately  12  miles  by  12  miles)  was 
chosen  as  the  basic  search  area.  Each  basic  search  area  is 
therefore  identifiable  by  the  AMS  map  coordinates  or  the  AMS 
map  area  name. 

In  proposed  reference  systems,  information  about  each  roll  of 
imagery  recei ved--or  stored--would  be  encoded  for  manual  or  computerized 
access.  Two  feasible  methods  of  encoding  area  information  in  a  index  of 
prior  cover  were  considered  feasible  in  a  TIIF.  One  method  uses  a  graphic 
plot  of  the  image  area  coverage  of  each  mission  as  the  primary  index  data 
(see  Figure  l) .  This  plot  would  be  keyed  to  a  map  base.  The  other  method 
represents  imagery  coverage  by  letters  and/or  numbers.  A  relatively  fine 
grid  or  matrix  whose  elements  are  characterized  in  some  systematic  fash¬ 
ion  by  a  series  of  letters  and  numbers  is  superimposed  on  a  map  area. 

Image  coverage  is  represented  on  an  index  record  by  letter/nunber  combi¬ 
nations  of  grid  areas  covered  by  the  imagery.  Figure  2  illustrates  this 
type  of  index  record. 


The  present  experiment  was  conducted  to  evaluate  the  two  methods 
for  speed  and  accuracy  in  locating  rolls  of  referenced  imagery  in  a  TIIF. 


TIIF  INDEXING  TASKS 

Three  primary  tasks  must  be  performed  in  relation  to  an  index  in  an 
image  interpretation  system:  (l)  indexing  the  imagery,  (2)  searching 
the  index  for  information  about  the  content  and  location  of  the  stored 
imagery,  and  (5)  updating  the  index  as  better  or  never  imagery  becomes 
available.  The  search  task  is  especially  Important  because  of  the  cri¬ 
tical  need  for  rapid  location  of  reference  materials  during  Hot  and 
Immediate  reporting  by  TIIF  personnel.  There  is  also  need  for  fairly 
rapid  access  to  reference  materials  for  orientation  prior  to  a  mission. 
Both  initial  indexing  and  updating  will  probably  be  accomplished  during 
lull  periods  in  a  TIIF.  The  search  task  has  therefore  been  singled  out 
as  the  focus  of  the  present  study. 

The  search  task  begins  when  the  interpreter  receives  a  requirement 
to  determine  if  previously  obtained  and  stored  imagery  contains  infor¬ 
mation  pertinent  to  the  present  mission  or  a  requirement  to  detect 
changes  of  military  significance  between  newly  acquired  imagery  and 
imagery  in  storage.  The  requirement  may  cone  from  G-2  or  the  local 
officer-in-charge  or  non-commissioned  officer-in-charge.  The  require¬ 
ment  is  incorporated  on  a  nap  overlay,  if  an  overlay  has  not  already 
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Figure  1.  Example  of  a  Graphic  Index  Record. 
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Figure  2.  Example  of  a  Map  Grid  and  Corresponding  Textual  Index  Record 


been  prepared.  The  interpreter  then  searches  through  his  prior  coverage 
index  to  identify  a  mission  whose  coverage  will  enable  him  to  answer  the 
requirement.  From  the  index,  he  obtains  the  accession  number(s)  of  the 
desired  roll(s)  of  imagery.  Using  the  accession  number(s),  he  retrieves 
the  roll(s)  of  film  from  storage. 


SPECIFIC  OBJECTIVES 


The  major  objective  of  the  present  study  was  to  compare  experimen¬ 
tally  two  indexes,  one  graphic  and  the  other  textual,  which  are  used  by 
an  image  interpreter  to  retrieve  tactical  imagery  for  specific  geograph¬ 
ical  areas.  Additional  objectives  were  to  investigate  how  a  specific  re¬ 
quirement  for  area  coverage  affects  the  use  of  indexes  by  an  interpreter.. 
Requirement  variables  included  size  of  an  area,  location  of  an  area  rela¬ 
tive  to  a  base  map,  and  whether  or  not  an  area  lies  on  two  adjoining 
maps.  Finally,  the  study  attempted  to  relate  certain  internal  character¬ 
istics  of  the  two  indexes  to  interpreter  performance. 


EXPERIMENTAL  METHOD 


Somple 


The  sample  comprised  l6  image  interpreters  Just  finishing  training 
at  Fort  Holabird  (1st  and  2nd  lieutenants)  and  KCOs  and  enlisted  men  as¬ 
signed  to  the  U.  3.  Army  Personnel  Research  Office  or  attached  to  the 
1st  Military  Intelligence  Battalion  at  Fort  Bragg.  Each  subject  worked 
through  each  indexing  problem  in  the  first  experiment  and  then  through 
each  indexing  problem  in  the  second  experiment. 

Experimental  Materials 

Graphic  Index.  'Thirty  mission  plots  and  their  corresponding  legends 
were  prepared  on  transparent  acetate  sheets.^  The  overlays  were  keyed 
to  the  map  of  Reagumgang  in  Korea,  Scale  1:50,000,  Army  Map  Service  (AM3) 
L751.  Twenty  mission  plots  were  prepared  on  transparent  acetate  sheets 
and  keyed  to  the  map  of  Kosong  in  Korea,  Scale  1:50, 000,  AMS  L75I,  which 
adjoins  the  Reagumgang  map.  The  order  of  missions  within  the  index  was 
random.  Figure  1  is  a  typical  graphic  index  record. 


^  Each  mission  was  plotted  on  a  separate  overlay.  This  procedure 

appeared  to  represent  a  likely  baseline  situation.  Another  method  was 
to  put  several  missions  on  a  single  overlay  and  color  code  each 
mission. 
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Textual  Index .  A  textual  index  was  prepared  using  the  sane  missions 
as  for  the  graphic  index.  Details  of  developing  the  textual  index  for¬ 
mat  for  presenting  area  coverage  are  discussed  in  Appendix  C.  Basically, 
the  grid  upon  the  index  record  represented  a  6  x  6  matrix  dividing  a 
1:50, 000  AMS  nlap  into  $6  equal  major  areas,  each  designated  by  a  capital 
letter  and  a  number  (Figure  2) .  For  the  map  grid,  each  of  these  areas 
was  further  divided  into  four  equal  areas  designated  by  the  small  letters 
a,  b,  c,  and  d.  The  textual  index  record  contained  this  grid,  except  that 
subareas  were  listed  in  a  single  row  of  four  units  instead  of  two  rows  of 
two  units.  Image  coverage  was  indicated  by  an  X  in  each  appropriate  grid 
area.  Figure  5  shows  a  textual  index  record  used  in  the  experiments. 

Where  there  was  overlap  coverage,  the  index  record  contained  the  words, 
"See  Also  Map  Coverage"  below  the  primary  grid,  as  well  as  a  display  of 
the  overlap  coverage. 

In  addition  to  the  textual  index  itself,  for  Experiment  2,  two 
special  forms  were  prepared  to  assist  the  subjects  in  using  the  index: 
a  transparent  index  record  to  help  select  relevant  index  records  and  a 
form  for  recording  each  selected  index  record's  coverage  (Appendix  C) . 

Problem  Areas.  Eight  areas  were  designated  as  problems  for  each 
experiment.  In  Experiment  1,  four  were  simply  points  and  four  were 
small  areas  (areas  smaller  than  a  single  grid  area) .  In  Experiment  2, 
all  the  areas  were  larger  than  a  single  grid  area;  four  areas  appeared 
on  a  single  map  and  four  areas  encompassed  two  adjoining  maps.  Each 
problem  area  is  described  further  in  Appendix  C. 

Experimental  Tasks 

Each  interpreter  serving  as  subject  was  provided  a  textual  index 
or  a  graphic  index.  He  was  then  given  a  series  of  problems  which  in¬ 
volved  searching  the  index  to  locate  imagery  coverage  for  designated 
points  or  areas.  Each  point  or  area  was  marked  on  an  overlay  attached 
to  an  1:50,000  AMS  map.  The  interpreter  was  informed  tnat  there  was  at 
least  one  correct  solution  and  that  he  was  to  record  the  first  solution 
he  found.  In  Experiment  2,  he  was  also  told  how  many  accession  numbers 
were  required  for  a  correct  solution. 

For  each  problem,  the  subject  first  checked  to  be  sure  he  under¬ 
stood  the  location  of  the  required  area  or  point  on  the  map  and,  in  the 
case  of  the  textual  index,  he  verified  the  alignment  of  the  grid  over  the 
map.  After  being  told  to  start,  he  searched  the  index  records  as  rapidly 
as  possible  for  an  index  record  or  records  which  provided  a  correct  so¬ 
lution  to  the  problem.  For  the  graphic  index,  this  search  normally  in¬ 
volved  aligning  five  or  six  index  records  (overlays)  simultaneously  ovr 
the  map  amd  determining  if  any  of  the  index  area  plots  covered  the  prob¬ 
lem  area.  When  a  solution  (or  partial  solution)  was  detected,  the  sub¬ 
ject  then  selected  the  correct  overlay  within  that  subset.  For  the 
textual  index,  the  subject  first  determined  which  map  grid  areas  were 
covered.  He  then  went  through  the  index  to  find  an  index  record  leaving 
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Figure  3.  Example  of  a  Textual  Index  Record. 
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coverage  for  those  grids.  Once  he  had  achieved  a  solution,  he  recorded 
the  accession  nucber(s)  on  a  recording  form  provided.  If  the  solution 
vas  incorrect,  the  experimenter  asked  the  subject  to  continue  to  search 
for  the  correct  solution. 

To  summarize,  the  experimental  tasks  of  the  subjects  were  to: 


1.  Verify  the  area  where  coverage  vas  required,  and,  in  case  of 
the  textual  index,  determine  what  grid  areas  the  problem  area  covered. 

2.  Search  through  the  index -records  until  a  record  (or  set  of 
records)  was  found  that  satisfied  the  problem  requirement. 


3.  Record  the  accession  number  of  the  selected  index  record(s)  on 
the  appropriate  form. 


Experimental  Variables 


The  primary  independent  variable  Jin  the  study  vas  the  method  of 
indexing  tactical  imagery.  The  two  indexing  methods,  designated  graphic 
(G)  and  textual  (T) ,  have  been  briefly  described.  The  methods  were  com¬ 
pared  under  a  variety  of  conditions.  An  analysis  of  situations  that 
might  be  encountered  using  an  area-oriented  imagery  index  indicated  that 
the  following  conditions  should  be  manipulated  or  controlled  in  com¬ 
paring  the  two  methods : 


1.  Location  of  problem  area.  The  location  of  the  problem  area  on 
a  map  might  be  important,  particularly  as  some  locations  might  have  more 
or  less  photo  coverage  than  other  locations. 

2.  Size  of  problem  area.  The  search  for  coverage  for  a  point 
(i.e.,  a  single  target)  might  be  different  than  for  a  large  area. 

3.  Problem  area  overlap.  The  base  map  covered  an  area  which 
appeared  likely  to  include  the  primaryl  area  of  interest  of  a  TUP. 
However,  if  primary  concern  vas  a  location  at  or  near  the  boundary  of 
the  nap  area,  a  problem  area  requirement  might  involve  searching  more 
than  one  map.  This  factor  appeared  important  since,  in  the  case  of  a 
problem  area  covering  two  maps,  the  search  task  had  to  include  index 
records  keyed  to  both  map  areas. 

4.  Amount  and  type  of  image  coverage  available.  Obviously,  the 
size  (number  of  index  records]  and  content  of  the  index  itself  is  an 
important  variable  which  affects  the  interpreter's  pertormar.ee  in 
searching  an  index.  For  example,  the  larger  the  index  or  the  smaller 
the  scale  of  the  imagery,  the  more  likely  the  user  of  the  index  is  to 
find  a  solution  to  a  problem. 
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The  only  dependent  variable  used  was  time  in  seconds  to  achieve  a 
correct  solution.  (Preliminary  trials  indicated  that  too  few  errors 
would  be  made  in  the  search  task  to  warrant  statistical  treatment.  A 
record  was  kept  of  the  errors,  however.)  Time  scores  which  are  obtained 
without  a  cut-off  limit  are  typically  skewed;  i.e.,  there  are  a  few 
scores  that  are  extremely , large  which  tend  to  distort  the  means.  This 
was  found  to  be  true  in  the  indexing  study.  Therefore,  prior  to  analysis, 
a  set  of  logarithmic  scores  ^  was  obtained  frcm  the  raw  scores. 

Experimental  Design 


3ecause  of  differences  in  the  basic  search  task  between  small-area 
and  large-area  problems,  especially  in  the  case  of  the  textual  index, 
it  was  decided  to  conduct  two  separate  experiments.  However,  the  de¬ 
sign  and  statistical  plans  for  Experiments  1  and  2  were  identical.  In 
Experiment  1,  the  two  indexes  were  compared  using  problems  with  small 
areas  or  point  locations  for  which  image  coverage  was  desired,  while  in 
Experiment  2  the  methods  were  compared  using  problems  with  large  areas 
some  of  which  covered  two  maps. 

A  Latin  Square  design,  shown  in  Figure  4,  was  used  in  both  experi¬ 
ments.  The  basic  design  was  replicated  using  two  sets  of  8  subjects  in 
each  experiment.  The  second  set  of  subjects  (9  through  16)  foil'  ed  the 
same  sequences  as  subjects  1  through  8.  There  were  two  groups  within 
each  set  of  8  subjects.  The  first  group  took  the  graphic  index  first, 
while  the  second  group  took  the  textual  index  first. 


Dofa  Collection  Procedures 

The  experimental  data  were  collected  in  the  U.  S.  AFRO  surveillance 
systems  laboratories,  at  Fort  Holabird,  and  at  Fort  Bragg.  The  subjects 
were  tested  individually  by  the  experimenter.  Each  subject  went  through 
the  entire  test  period  at  one  sitting  (about  three  hours) .  Each  experi¬ 
mental  session  was  organized  as  follows: 

General  Explanation  of  Test 
Experiment  1 

Specific  instructions  for  first  indexing  method 


^  The  logarithmic  scores  were  calculated  by  the  formula: 

[  (Log10  x)-l  ]  x  100  =  x 

One  was  subtracted  from  the  log  score  and  the  result  was  then 
multiplied  by  100  in  order,  to  obtain  numbers  which  could  be  easily  ma¬ 
nipulated  in  the  analyses.  Paw  scores  are  shown  in  Appendix  A. 
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Group  1 


uroup  2 


Method  of  Indexing 

Subject  Graphic  Index  (G)  Textual  Index  (T) 


S1 
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3 
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8 

S2 
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S3 
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S6 
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S7 
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5 

S8 
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1  ! 
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6 

Trials 

I 

II  , 

Hi 

IV 

V 

i 

VI 

j 

_ 

VII 

VIII 

Note  : 

Problems  3>  4 >  6,  and  8  are  Point  (P)  problems  and  1,  2,  5  and  7  are 
Gmail  area(s)  problems  in  Experiment  1. 

Problems  1,  3,  5,  and  7  are  No  overlap  (N)  problems  and  2,  k,  6,  and  8 
are  Overlap  (0)  problems  in  Experiment  2. 

In  Group  1  above,  a  block  of  graphic  cond it  ions  occurred  before  a  block  of 
textual  conditions. 

In  Group  2  above,  a  block  of  textual  cond it  ions  occurred  before  a  block  of 
graphic  conditions. 

The  above  design  was  replicated  for  Subjects  9-16. 


r  igure 
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Training  on  above 
Testing  on  above 


Repeat  for  other  indexing  method 
Experiment  2 

Specific  instructions  for  first  indexing  method 
Training  on  above 
Testing  on  above 


Repeat  for  other  indexing  method 

A  data  collection  fo!rm  was  provided  each  subject  on  which  to  record 


accession  numbers  (Figure 
ing  data  about  each  subje; 


p-7) .  The  experimenter  had  a  form  for  record- 
(Sct  and  the  time  taken  to  solve  each  problem. 

The  experimenter  measured  time  using  a  stop  watch  and  recorded  any 
errors  made  by  the  subject  (Figure  C-8) .  Time  was  measured  in  both 
experiments  from  the  moment  a  subject  received  the  map  with  the  problem 
area  on  it  until  he  recorded  the  correct  accession  number(s).  Once  any 
solution  was  obtained,  the  watch  was  stopped.  If  the  solution  was  in¬ 
correct,  the  subject  was  informed  and  the  stop  watch  restarted. 


RESULTS 


The  analysis  of  variance  solutions  for  the  log  time  scores  of 
Experiments  1  and  2  are  shown  in  Tables  1  and  2.-^  In  both  experiments, 
significant  variance  ratios  were  obtained  for  both  the  method  of  index¬ 
ing  and  the  problems  within  area  and  problems  within  overlap. 


^  The  e  term  in  Tables  1  and  2  was  used  to  test  al T  within  subject 

factors  except  the  area  factor  in  Experiment  1  and  the  overlap  factor 

in  Experiment  2.  In  these  two  cases,  e  was  thought  to  be  the  more 

2 

appropriate  error  term  since  these  factors  are  random  factors,  i.e., 
they  sample  only  the  range  of  the  overlap  or  area  dimensions.  The  e 

3 

term  in  the  between  subjects  part  of  the  analyses  was  used  to  test 
sequences  within  groups  factor  which,  when  found  not  significant,  was 
designated  e  and  used  to  test  the  final  between  subjects  factor. 

4 
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Table  1 


EXPERIMENT  1:  ANALYSIS  OF  VARIANCE  CF  LOG  TIME  SCORES 


Source  of  Variation 

Degrees 

of 

Freedom 

Sum  of 
Squares 

Mean 

Square 

F 

Between  Subjects 


f-equences 

Groups 

1 

147 

147.OO 

O.27 

Sequences  v/Groups  (e^) 

6 

3,167 

527.83 

2-77 

Subjects  w/3equences  (e^) 

8 

1,524 

190.50 

Within  Subjects 

Trials 

Blochs  of  Trials 

1 

193 

193-00 

0.45 

Trials  v/Blocks 

6 

3,956 

659.33 

1.48 

Problems 

Point  versus  Area  Factor 

1 

6,258 

6258.00 

2.45 

Problems  v/Area  (e^) 

6 

15,336 

2556.00 

.  5.74* 

Method  of  Indexing 

1 

4,313 

4515.00 

9.69* 

Latin  Square  Error  plus 

Subjects  w/3equences  x 

Trials  (e^) 

TOTAL 

97 

127 

43,155 

444.90 

*F  ratio  significant,  P  <  .01 
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Table  2 


EXPERIMENT  2:  ANALYSIS  OF  VARIANCE  CF  LOG  TIME  SCORES 


Source  of  Variation 

Degrees 

of 

Freedom 

Sum  of 
Squares 

Mean 

Square 

F 

Between  Subjects 

Sequences 

Groups 

1 

18 

18 

0.01 

Sequences  v/Groups  (e^) 

6 

6,774 

1,462 

4.24 

Subjects  w/Sequences  (e^) 

8 

2,757 

545 

Within  Subjects 
Trials 


Blocks  of  Trials 

1 

2 

2 

0.00 

Trial.,  w,  ■ . .  0  ks 

6 

4,113 

686 

1.30 

Problems 

Overlap  Factor 

1 

2,775 

2,775 

1.10 

Problems  w/Overlap  (e^) 

6 

15,117 

2,520 

4.79* 

Method  of  Indexing 

1 

54,126 

34,126 

64.88* 

Latin  Square  Error  plus 

Subjects  w/Sequences  x 

Trials  (e.^B 

TOTAL 

97 

127 

51,048 

526 

*F  ratio  significant,  P  <  .01 


1  The  Latin  Square  error  was  significant  (F  =  4.57)  when  the  tern  used  to 
test  the  error  term  was  subjects  w/'sequences  x  trials.  From  an  informal 
analysis  of  the  error  term,  this  result  was  considered  due  to  an  inter¬ 
action  between  problems  and  method  of  indexing  caused  by  the  deviation 
of  Problem  3  from  the  general  treat  (dee  Table  4) .  However,  when  using 
the  pooled  error  tens  as  in  Experiment  1,  the  F’s  for  method  of  indexing 
were  very  significant  and  appeared  significant  for  the  nroblens  within 
overlap  factor.  Since  these  results  agree  with  Expo rim  .  .  and  because 

of  the  magnitude  of  the  P's,  the  differences  were  tr  -1  significant 

though  caution  is  warranted  because  of  the  invalid  error  tc. m. 
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Method  of  Indexing 


Tables  3  and  ^  present  the  log  time  means  for  the  two  indexing 
methods  for  each  problem  in  both  experiments.  Although  these  means  are 
significantly  different,  the  direction  of  the  difference  is  not  the  same 
in  the  two  experiments.  In  Experiment  1,  the  textual  means  for  the  prob¬ 
lems  are  smaller  than  the  graphic  means  while  in  Experiment  2,  the  graphic 
means  are  smaller  than  the  textual  means.  This  result  may  be  explained 
as  follows :  With  the  textual  index  in  Experiment  1,  the  subject  had 
merely  to  remember  a  simple  designator,  composed  of  three  characters--(e .g. , 
F4b)  --or  four,  if  the  problem  fell  across  two  subarea  boundaries  (e.g., 

F4b  and  c) .  Keeping  these  designators  in  mind,  a  subject  could  quickly 
scan  the  textual  index  records  and  select  an  appropriate  record.  An  ex¬ 
ception  was  Problem  J.  This  problem  required  the  subject  to  remember  two 
different  grid  designators,  D4b  and  D3d,  since  the  problem  area  fell 
along  the  border  of  two  major  grid  areas.  Thus,  Problem  7  is  similar  to 
the  large  area  problems  in  Experiment  2  in  the  complexity  of  the  textual 
grid  requirements. 

In  the  case  of  the  graphic  index,  the  manipulation  of  the  large 
heavy  overlays  by  the  subjects  was  sufficiently  cumbersome  to  cause  the 
time  scores  to  rise  above  those  of  the  same  problems  using  the  textual 
index  as  indicated  by  the  mean  scores  in  Experiment  1.  However,  once  the 
textual  index  became  sufficiently  complex  that  a  subject  could  not  re¬ 
member  the  grid  requirements  and  had  to  use  the  transparent  index  record 
and  the  recording  forms  to  utilize  the  index  as  in  Experiment  2,  the  time 
scores  for  the  textual  index  increased  above  those  for  graphic  index. 

In  Experiment  2,  note  that  Problem  3  reverses  sharply  the  general  trend 
for  the  textual  index.  Problem  3  approached  the  simplicity  of  problems 
in  Experiment  1  in  that  it  required  but  one  major  grid  area  designator 
even  though  it  covered  all  that  grid;  i.e.,  the  subject  had  only  to  look 
for  "all,  of  F6,"  rather  than  "F6a,  F6b,  F6c,  and  F6d." 

The  natter  of  precision  also  contributed  to  the  differences  between 
the  two  methods,  especially  in  Experiment  1  where  the  problem  areas  were 
smaller  than  the  grid  areas.  Using  the  textual  index,  the  subject  was 
to  locate  mission  coverage  for  all  the  grid  areas  that  were  partially  or 
completely  covered  by  the  problem  areas.  For  Experiment  1,  a  solution 
could  be  obtained  for  a  problem  area  even  though  the  mission  selected 
did  not  cover  the  problem  area  if  the  mission  covered  the  grid  suffi¬ 
ciently  (25^  or  more)  to  cause  the  grid  to  be  marked  with  an  X,  This 
precision  would  not  be  as  critical  in  Experiment  2,  as  the  problem 
are -3  were  large  compared  bo  the  grid  areas  and  the  problem  areas 
usually  covered  good  portions  of  all  the  grid  areas  affected.  The 
choice  of  grid  size  for  a  textual  Index  is  important  and  should  be 
studied  further.  If  precision  is  required,  then  a  graphic  index  may  be 
superior  to  a  textual  index  'unless  the  latter  has  a  grid  whpse  elements 
are  considerably  smaller  than  the  problem  areas  likely  to  be  encountered 
when  using  it. 
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EXPERIMENT  1:  LOG  TIME  MEANS  FOR  GRAPHIC  AND  TEXTUAL  INDEXES 


Problem 

Number 

Problem 

Type 

Graphic  Textual  Total 

1 

Sm.  Area 

51.0 

49.8 

50.4 

2 

Sm.  Area 

56.9 

3C  5 

43.7 

3 

Point 

58. 0 

27.9 

42.9 

4 

Point 

50.9 

32.2 

4l.6 

5 

Sm.  Area 

57.4 

58.9 

58.1 

6 

Point 

62.8 

44.5 

53.6 

7 

Sm.  Area 

72.6 

91.8 

82.2 

8 

Point 

49.8 

30.9 

40.3 

TOTAL 

57.4 

45.8 

Table  4 


EXPERIMENT  2: 

LOG  TIME 

MEANS  FOR 

GRAPHIC  AND 

TEXTUAL  INDEXES 

Problem 

Number 

Problem 

Type 

Graphic 

Textual 

Total 

1 

No 

Overlap 

90.6 

158.6 

124.6 

2 

Overlap 

109.4 

162.1 

135.8 

3 

No 

Overlap 

130-8 

96.0 

113.4 

4 

Ct/e  r  lap 

121.2 

133.8 

127.5 

5 

No 

Overlap 

111.0 

171.9 

141.4 

6 

Overlap 

109.6 

129.0 

119.3 

7 

No 

Overlap 

83.I 

137.8 

110.4 

8 

Overlap 

130.6 

158.5 

144.6 

TOTAL 

110.8 

143.5 

Areas  on  whn.ii  P < ! em v  v.cr-  Based 

The  analysis  for  both  experiments  indicated  that  problem  areas  were 
significantly  different  among  themselves.  However,  the  point  versus  area 
factor  in  Experiment  1,  and  the  overlap  factor  in  Experiment  2,  did  not 
produce  these  differences  (See  Tables  1  and  2).  Although  the  problem 
areas  in  the  two  experiments  varied  in  size,  overlap,  and  in  location  on 
the  base  map,  the  factor  that  probably  contributed  most  to  the  differences 
among  problems  was  the  content  of  the  indexes.  In  the  case  of  the  graphic 
index,  a  problem  area  sometimes  lay  directly  under  the  path  of  a  flight 
line  containing  small  scale  photo  coverage  and  could  be  quickly  detected. 
In  other  cases,  the  scade  of  the  coverage  was  larger,  and  therefore  the 
index  record  was  more  difficult  to  recognize  as  being  a  solution.  In 
some  cases,  it  was  obvious  that  only  two  or  three  index  records  could 
possibly  contribute  to  u.  solution,  while  in  other  cases  a  large  propor¬ 
tion  of  the  index  records  covered  the  general  area,  and  the  subjects  had 
to  go  through  a  large  number  before  finding  the  one  he  wanted. 

Another  factor  contributing  to  the  differences  among  problem  areas 
in  Experiment  2  was  the  large  variation  in  the  number  of  grids  to  be 
covered,  which  resulted  in  corresponding  difficulty  in  using  the  index 
record  and  recording  forms.  Finally,  depending  upon  the  coverage  pat¬ 
tern  on  a  textual  index  record  in  Experiment  2,  the  required  areas  might 
be  detected  either  easily  or  with  difficulty.  If  the  general  area  was 
completely  covered  with  X's  or  if  only  the  required  areas  were  covered 
with  X's,  it  was  fairly  simple  for  the  subject  to  detect  a  solution.  If 
the  general  area  contained  many  blanks  and  X's  which  alternated  with 
each  other  forming  a  checkered-like  pattern,  it  was  more  difficult  to 
detect  a  solution,  even  in  Experiment  2  when  a  transparent  template  was 
used  to  help  simplify  this  situation. 

Other  Variables 

The  means  for  trials,  groups,  and  sequences  are  presented  in  Table  5* 
In  both  experiments,  there  were  no  significant  differences  among  trial 
means,  indicating  no  practice  effect  during  the  experimental  sessions. 

(The  subjects  had  received  considerable  training  prior  to  testing.) 

There  were  no  differences  in  groups  associated  with  the  sequence  of  tex¬ 
tual.  and  graphic  methods. 

Error  Scores 


Relatively  few  errors  were  made  by  the  experimental  subjects  in 
accomplishing  the  problems.  In  all,  the  16  subjects  produced  256  scores 
in  the  two  experiments.  Errors  occurred  in  only  19  instances,  broken 
down  as  follows: 

Graphic  Textual 

Experiment  11  3 

Experiment  2  5  10 
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There  were  more  than  twice  as  many  errors  with  the  textual  index  as 
with  the  graphic  index.  There  were  more  than  twice  as  many  errors  in 
Experiment  2  as  in  Experiment  1.  These  data  support  the  time  score  re¬ 
sults  which  showed  that  the  tasks  in  Experiment  2  were  the  more  difficult 
and  that  the  subjects  used  the  graphic  index  more  readily  for  large  area 
problems . 


Table  5 

LOG  TIME  MEANS  FOR  TRIALS,  SEQUENCES,  AND  GROUPS 


Trial 

Means 

Experiment  1 

Sequence 

Means 

Group 

Means 

Trial 

Means 

Experiment  2 

Sequence 

Means 

Group 

Means 

I  58.8 

1 

56.6 

1  52.7 

1 

123.6 

1 

144.2 

1  126.8 

II  41.8 

2 

53-3 

2  50.7 

11 

CO 

3 

2 

119.4 

2  127.5 

III  56.4 

3 

51-5 

III 

129.6 

3 

126.4 

IV  44-5 

4 

49.3 

IV 

131.1 

4 

116.9 

v  55.8 

5 

60.2 

V 

129.1 

5 

126.1 

vi  48.6 

6 

51-2 

VI 

115.4 

6 

134.8 

vii  51.8 

7 

48.5 

VII 

135-9 

7 

127.6 

Vlii  55.1 

8 

42.1 

VIII 

127-5 

8 

121.5 
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CONCLUSIONS  AND  RECOMMENDATIONS 


The  following  conclusions  ere  based  on  the  results  presented  in  the 
preceding  section: 

1.  In  general,  the  type  of  requirement  or  request  that  will  be 
imposed  upon  an  index — the  kinds  of  questions  that  are  likely  to  be 
asked  of  an  index--is  important  in  the  choice  of  a  manual  indexing 
method  for  a  tactical  image  processing  system. 

2.  A  graphic  index  for  depicting  tactical  imagery  stored  on  roll 
film  is  superior  to  a  textual  index. if  the  areas  or  targets  for  which 
coverage  is  required  are  large  or  if  precision  is  requii'ed  when  selecting 
imagery  coverage. 

3.  A  textual  index  for  depicting  tactical  imagery  stored  on  roll 
film  is  superior  to  a  graphic  index  if  the  areas  for  which  coverage  is 
desired  are  small  or  if  the  image  coverage  does  not  have  to  be  precisely 
defined. 


The  following  recommendations  are  made  with  regard  to  future  efforts 
in  this  area: 

1.  The  textual  index  concept  should  be  further  investigated  for 
use  in  a  computerized  TIIF  where  a  variety  of  index  listings  can  easily 
be  provided.  In  particular,  studies  should  be  made  of  the  fineness  of 
the  grid  which  is  used  to  quantify  the  area,  the  format  of  the  portion 
of  the  index  record  which  depicts  area  coverage,  and  the  development  of 
computerized  or  other  aids  to  relieve  the  interpreter  of  any  bookkeeping 
tasks  associated  with  using  a  textual  index. 

2.  The  graphic  index  record  format  should  be  studied  with  a  view 

to  improving  the  pictorial  representation  of  the  area  covered,  evaluating 
the  effect  of  displaying  several  missions  on  a  single  overlay,  and  sim¬ 
plifying  the  task  of  searching  the  index  records. 

3.  A  combination  of  graphic  and  textual  indexing  methods  should  be 
considered.  A  textual  index  might  be  useful  as  a  filter  or  screening 
technique  for  selecting  a  subset  of  graphic  index  records  which  could 
then  be  searched  to  obtain  precise  coverage  over  a  specified  area.  Be¬ 
sides  providing  manual  back-up  materials  in  case  of  equipment  malfunc¬ 
tion,  the  combination  approach  in  a  computerized  TIIF  would  allow  selec¬ 
tion  of  previous  missions  on  source  data  (date,  sensor)  as  well  as  area 
coverage.  The  TIIF  computer  could  perform  the  search  of  the  textual 
index. 
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APPENDIX  B 


ORIENTATION  OF  SUBJECTS  IN  INDEXING  EXPERIMENTS 


i:;structio:;s  to  subjects:  . 

"The  test  that  you  are  about  to  take  is  to  help  evaluate  two  ways 
of  indexing  tactical  imagery  cover  stored  on  roll  filn.  One  r.ethod  is 
called  a  graphic  r.ethod  and  is  sinilar  to  existing  methods  used  by  the 
military  services  where  photo  coverage  is  plotted  on  overlays  which  are 
keyed  to  a  base  map  [Experimenter  shows  the  subject  a  graphic  index 
record  and  a  r.ap] .  The  other  method  is  a  textual  index  where  geographic 
areas  arc  described  in  terms  of  coded  crxa&rants  which  represent  areas  of 
image  coverage  [Experimenter  shows  subject  a  textual  index  record]. 

These  coded  areas  are  also  keyed  to  a  base  map.  Each  index  record,  in 
both  types  of  indexes,  represents  a  single  mission  on  a  single  roll  of 
film.  The  scale  of  the  imagery  varies  from  about  1:2500  to  1:7500  and 
is  all  vertical  photography.  The  photo  coverage  for  the  graphic  method 
is  directly  depicted  on  an  overlay  by  means  of  area  plots  every  fifth 
frame  and  for  the  textual  method  is  represented  by  check  marks  (X's) 
in  cells  of  the  index  records.  [Experimenter  shows  subjects 
similar  textual  and  graphic  index  records].  V.'hen  the  photo  coverage 
covers  more  than  a  single  map  area,  you  will  find  a  dotted  line  on  the 
graphic  index  record  indicating  that  there  is  an  overlay  keyed  to  an  ad¬ 
joining  map  with  tine  remaining  coverage,  or  there  will  be  a  "lee  Also 
Map  Coverage'  indication  at  the  bottom  of  the  textual  index  record.  You 
will  need  to  use  this  information  when  your  problem  involves  finding 
coverage  for  two  map  areas.'' 

"The  test  is  divided  into  two  major  phases;  the  first  test  phase 
compares  the  two  types  of  indexes  using  small  areas  and  points  on  a 
single  map  [Experimenter  points  to  map  of  k'aegumgang]  and  the  second 
test  compares  the  indexes  using  large  areas  and  on  occasion  two  maps 
L Experimenter  points  to  both  the  "ae-gumgang  map  and  the  Ko song  map]. 

In  the  first  test,  you  will  be  given  training  on  four  problems  using  one 
of  the  index  methods.  Some  of  you  will  start  with  the  graphic  index, 
while  others  will  start  with  the  textual  index.  Once  you  have  worked 
the  four  training  problems  and  have  demonstrated  that  you  ’understand  the 
procedure,  you  will  be  given  fo^ur  test  problems,  one  at  a  time,  using 
that  same  type  of  index.  I  will  tell  you  to  start  using  the  index  by 
saying  "Start."  At  this  time,  I  will  start  a  stopwatch  and  you  will  be¬ 
gin  to  search  through  the  index.  Once  you  have  recorded  your  answer 
(accession  number)  on  the  recording  sheet,  I  will  stop  the  watch  and 
check  your  solution  against  the  answer  list.  If  you  are  correct,  I  will 
indicate  this,  and  we  will  proceed  to  the  next  problem.  If  you  are  in¬ 
correct,  I  will  tell  you  this,  and  you  will  continue  to  work  at  the  same 
problem  until  you  obtain  another  solution.  I  will  restart  the  stopwatch 
when  you  restart  working  the  problem.  This  procedure  will  continue  until 
you  obtain  a  correct  solution  to  the  problem." 
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"Once  the  four  test  problems  are  completed.,  you  will  be  trained  on 
the  sane  four  training  problems  using  the  other  indexing  method.  Again, 
after  completing  the  training  problems,  you  will  be  given  four  more  new 
test  problems  using  the  second  indexing  method.  Are  there  any  questions? 
[Experimenter  answers  questions.]  We  will  now  start  the  first  test 
phase . " 


The  following  instructions  were  given  subsequent  to  the  completion 
of  Experiment  1. 

"We  are  now  ready  to  begin  the  second  part  of  the  testing.  As  be¬ 
fore,  you  will  be  given  eight  problems,  four  involving  each  type  of  in¬ 
dex.  Prior  to  the  testing  on  each  type  of  index,  you  will  work  two 
training  problems.  You  will  use  the  index  in  the  same  order  as  you  did 
before.  The  major  difference  between  what  you  will  do  now  and  what  you 
did  before  is  that  you  will  be  using  problems  that  are  larger  in  area, 
occupy  more  than  one  map  area  in  some  cases,  and  require  two  or  three 
accession  numbers  for  a  solution." 


Instructions  for  the  Graphi c  Indc  in  Experiment  2 . 

"The  graphic  index  for  this  test  involves  two  sets  of  overlays-- 
one  geared  to  the  base  map,  r.'aegumgang,  which  you  used  before,  and  one 
set  geared  to  the  adjoining  map,  rCosong.  Flights  or  missions  that  in¬ 
clude  both  maps  have  dotted  lines  at  the  border  of  the  overlay  and  have 
identical  accession  numbers.  If  the  problem  involves  a  single  map,  you 
will  operate  pretty  much  as  you  did  before.  The  only  difference  is  that 
more  than  one  overlay  will  be  required  in  order  to  completely  cover  an 
area.  Gince  it  is  the  right  combination  of  overlays  that  is  important, 
you  will  be  told  whether  you  will  need  two  or  three  accession  numbers  for 
a  solution.  You  need  to  keep  track  of  those  index  records  which  contri¬ 
bute  some  coverage  until  you  get  the  complete  solution.  This  can  be  done 
by  leaving  these  overlays  on  the  map  or  in  a  special  place  close  to  you. 
As  soon  as  you  obtain  a  complete  solution  with  the  correct  number  of  ac¬ 
cession  numbers,  record  the  solution  on  the  recording  form." 

"For  problems  involving  two  map  areas,  a  good  procedure  is  to  solve 
the  problem  for  the  prime  map  area  (Naegumgang)  first.  Then,  if  any  of 
the  index  records  have  dotted  lines  indicating  overlap  coverage,  go  to 
those  same  accession  numbers  in  the  index  records  for  the  adjacent  map 
to  see  if  those  records  solve  the  area  requirement  there  also.  Usually, 
these  records  will  provide  at  least  some  of  the  required  coverage.  Then 
you  can  proceed  to  obtain  the  rest  of  the  solution. _  Again,  you  will  be 
told  how  many  accession  numbers  will  be  required  for  the  solution.  Also 
remember  that  index  records  having  coverage  in  adjoining  map  areas  have 
only  one  accession  number.  Are  there  any  questions?  [Experimenter 
answers  questions.]  Now  we  will  work  the  two  training  problems." 
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Instructions  for  the  Textual  Index  in  Experiment  2. 

"The  textual  index  for  this  test  involves  a  set  of  records  for  each 
of  the  two  maps.  However,  any  coverage  that  falls  into  an  adjoining  map 
areas  is  included  in  the  cross-reference  section  at  the  bottom  of  a 
record.  The  index  records  for  the  primary  map,  Naegumgang,  include  the 
30  missions  over  that  map  area  as  well  as  15  missions  that  are  continued 
into  the  adjacent  map  area  of  Kosong.  Consequently,  the  primary  set  of 
textual  index  records  contains  information  on  4  5  of  the  rjQ  area  plots. 

As  you  might  expect,  most  problems  can  be  completely  solved  using  Just 
the  primary  index  set.  However,  if  you  do  need  coverage  unique  to  the 
adjoining  map  area,  you  can  check  those  index  records." 

"Because  it  is  difficult  to  remember  al I  of  the  grid  areas  that  are 
required  to  solve  these  more  difficult  problems,  two  items  have  been  pro¬ 
vided  to  assist  you  in  searching  the  records  and  determining  the  correct 
accession  numbers.  The  first  of  these  items  permits  you  to  record  all 
the  grid  areas  for  which  you  want  to  get  image  coverage.  [Experimenter 
shows  the  textual  index  record  accession  number  recording  form  illustra¬ 
ted  in  Figure  C-3 * ]  The  primary  grid  characters  (e.g.,  A3)  are  written 
in  the  top  row,  and  the  subarea  characters  (e.g.,  b)  are  written  in  the 
second  row.  How,  as  you  go  through  the  index,  for  each  index  record  that 
has  coverage  you  record  an  X  in  the  appropriate  column  or  columns  and  the 
corresponding  accession  number  at  the  left.  You  continue  in  this  fashion 
until  you  have  coverage  for  all  the  grid  areas  indicated  at  the  top  using 
the  correct  number  of  accession  numbers,  either  2  or  3-  Once  you  have 
the  solution,  you  record  the  accession  numbers  of  the  solution  index  records 
on  the  answer  form." 

"In  addition,  there  is  another  form  that  will  be  helpful  to  you. 

This  is  a  transparent  acetate  template  which  contains  a  blank  index 
record  form  upon  it.  [Experimenter  shows  this  item  to  the  subject.] 

After  you  have  recorded  the  grid  areas  you  must  search  for,  you  should, 
using  this  grease  pencil,  indicate  these  areas  on  the  template.  You  can 
do  this  by  outlining  each  of  the  boxes  or  putting  an  X  in  each  box.  Nov, 
when  you  think  an  index  record  may  have  coverage  for  your  problem,  you 
place  this  template  over  the  index  record  sheet  and  determine  what  grid 
areas  are  covered.  This  simplifies  having  to  study  each  index  record. 

Are  there  any  questions?  [ Experimenter  answers  the  questions.]  All 
right,  now  we  will  work  the  two  training  problems  using  the  textual 
index  and  these  aids." 
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APPENDIX  C 


DEVELOPMENT  OF  EXPERIMENTAL  MATERIALS  AND  RECORDING  FORMS 


GRAPHIC  INDEX 

The  graphic  index  consisted  of  a  series  of  transparent  overlays 
which  depicted  the  boundries  of  the  coverage  of  the  imagery.  (3ee 
Figure  1  in  text) .  Each  mission  was  plotted  on  a  separate  overlay  and 
each  overlay  was  keyed  to  a  base  1:50,000  AMS  map.  Each  overlay  (area 
plot  and  legend)  was  an  index  record,  and  a  set  of  overlays  representing 
all  the  missions  was  the  graphic  index.  The  legend  on  each  overlay  con¬ 
tained  source  data  about  the  mission  such  as  the  name  of  the  image-taking 
organization,  the  date  of  the  mission,  and  the  approximate  scale.  Figure 
C-l  shows  a  legend  containing  source  data  based  on  one  of  the  overlays. 
However,  only  the  accession  number  and  the  reference  to  the  base  map  were 
used  in  the  experiment. 

Since  a  single  photo  mission  may  cover  more  than  one  base  map, 
many  of  the  records  had  dashed  lines  at  the  border  indicating  that  there 
was  additional  coverage  in  an  adjoining  map  area  (Figure  l) .  A  separate 
overlay  containing  the  coverage  keyed  to  the  adjoining  map  had  the  same 
accession  number,  since  it  represented  the  same  mission. 

Thirty  index  records  were  developed  for  the  map  Naegumgang  of 
Korea,  Scale  1:50,000,  Army  Map  Series  (AM3)  L7%,  Sheet  6 <029  IV  and 
twenty  index  records  for  the  map  Kosong  of  Korea,  Scale  1:50,000,  AMS 
L751,  Sheet  68291;  the  latter  map  area  adjoins  the  former  map  area  along 
the  former's  eastern  boundary.  These  particular  raps  were  chosen  be¬ 
cause  severed,  copies  were  available  to  the  experimenter.  The  Naegumgang 
map  was  designated  as  the  primary  area  of  interest  for  the  experiments. 
Fifteen  of  the  thirty  index  records  for  Naegumgang  contained  missions 
that  overlapped  the  Kosong  map.  Thus,  the  Kosong  index  contained  15 
overlap  missions  and  five  unique  to  the  Kosong  area.  The  scale  of  the 
image  coverage  varied  from  approximately  1:2500  to  1:7500.  The  scale  of 
each  mission  was  indicated  in  the  legend  of  that  record.  To  provide 
realistic  mission  plots,  the  mission  plots  in  the  Standard  Indexing 
System  library  at  Aeronautical  Charting  and  Information  Center  in  Arling¬ 
ton,  Virginia  were  examined.  Several  of  the  index  records  developed  for 
the  experiments  contained  mission  plots  similar  to  plots  of  missions 
actually  flown  by  reconnaissance  aircraft. 

The  area  plot  on  each  overlay  insisted  of  a  series  of  individual 
plots  of  discrete  frames  of  imager;,  linked  by  a  straight  line.  The  size 
of  the  individual  frame  was  inversely  related  to  the  scale :  the  larger 
the  scale,  the  smaller  the  individual  plot.  The  line  connecting  the  in¬ 
dividual  plots  indicated  the  flight  line  of  the  aircraft.  The  imagery 
frames  were  plotted  about  every  fifth  frame  or  every  time  the  flight 
line  changed  direction.  Therefore,  the  actual  coverage  fell  within  two 
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Graphic  Index 
Record 


SENSOR  ID  Vert  Photo 

AREA  Korea,  1:50,000 

DATE  ?  May  196 5 

AMS  L751 

TAKING  ORG  IT  TP.S  -  565 

SHEET  66291 

APPROX.  SCADS  1:5800 

1 

ACCESSION  NO.  06c> 

Figure  C-l.  Legend  for  a  Graphic  Index  Record 
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parallel  lines  that  bounded  the  plotted  frames.  The  overlays  were 
transparent  sheets  of  acetate  whose  dimensions  were  18-1/2  by  20  inches.. 
The  plotting  area  itself  was  the  same  size  as  the  AMS  map  base--l4-l/2 
by  17  inches.  The  comer  alignment  narks  on  the  overlay  indicated  where 
the  overlay  was  to  be  aligned  with  the  map.  The  index  records  for  each 
map  area  were  stored  separately  and  were  unbound. 


TEXTUAL  INDEX . 

The  textual  index  records  were  developed  using  the  area  coverage  of 
the  graphic  index  records.  Some  of  these  textual  records  were  subse¬ 
quently  modified  in  order  to  equalize  the  number  of  index  records  re¬ 
quired  for  a  solution  to  a  specific  problem. 

The  grid  selected  for  the  textual  index  was  a  6x6  matrix  which 
divided  the  base  map  area  into  38  equal  areas  whose  locations  were 
identified  as  shown  in  Figure  C-2.  Each  area  was  further  subdivided  into 
four  equal  areas,  lettered  a  through  d,  proceeding  as  before  from  left 
to  right  and  from  bottom  to  top.  Thus,  each  l/l44  of  a  map  area  was  de¬ 
signated  by  three  characters  (e.g.,  B3c) .  This  grid  was  felt  to  be  suf¬ 
ficiently  precise  to  provide  a  test  of  validity  of  the  indexing  concept. 

The  grid  was  placed  over  each  of  the  graphic  overlays  to  determine 
what  grid  areas  were  covered  by  each  mission.  In  this  manner,  for  each 
overlay,  a  textual  index  record  was  generated.  For  a  grid  area  to  have 
coverage,  at  least  2571  of  the  grid  had  to  be  covered  by  a  graphic  plot. 
Obviously,  a  change  in  the  criterion  would  change  the  content  of  the 
index  record--the  higher  the  percentage  figure  used  as  the  criterion, 
the  less  likely  a  grid  would  be  selected.  Figure  3  in  the  text  of  the 
report  depicts  a  typical  textual  index  record  used  in  the  experiment. 

The  index  records  were  printed  on  standard  size  government  bond  paper. 

In  order  to  depict  overlap  of  areas  into  an  adjoining  map,  a  cross- 
referencing  scheme  was  utilized.  Whenever  a  mission  inducted  coverage 
in  an  adjoining  map,  a  "See  Also"  reference  to  the  appropriate  map  base 
was  inserted  with  a  print-out  of  the  coverage  of  that  map.  Because  of 
the  cross-referencing  feature,  the  textual  index  could  contain  more  in¬ 
formation  about  photo  missions  in  a  smaller  set  of  index  records  than 
the  graphic  index.  This  cross-referencing  was  included  because  it  is 
a  logical  characteristic  of  a  textual  indexing  system. 

As  in  the  case  of  the  graphic  index,  there  were  30  index  records 
for  the  primary  map  area  (Naegumgang)  of  which  15  contained  the  cross- 
reference  information.  The  secondary  map  area  (Kosong)  induded  20 
index  records  of  which  15  were  cross-referenced  back  to  the  primary  map 

area. 


With  the  textual  index,  the  subject  utilized  the  grid  which  was 
keyed  to  the  problem  map.  'When  working  a  problem,  the  subject  first 
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Figure  C-2.  Grid  Used  to  Generate  Textual  Index  and  Sample 
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determined  vhat  grid  areas  required  coverage.  In  the  case  of  the  problem 
area  in  Figure  C-2,  areas  F3c  and  FJd  require  coverage.  The  subject 
would  then  search  through  the  primary  index  until  he  located  the  index 
records  that  provided  complete  coverage  for  F3c  and  F3d.  Since  some  of 
the  problems  were  quite  large,  it  became  difficult  to  remember  all  the  re¬ 
quired  areas  as  veil  as  to  keep  trade  of  all  the  index  records  which  con¬ 
tributed  some  coverage  toward,  the  solution.  For  example,  eight  or  nine 
index  records  might  provide  some  coverage  for  a  problem  requiring  six 
small  grid  areas  for  a  solution.  Of  these,  two  or  three  might  do  the  com¬ 
plete  job,  but  the  right  combination  might  not  become  known  to  the  search¬ 
er  until  most  of  these  records  had  been  identified  as  partial  contributors. 
It  was  decided  to  divide  the  indexing  experiment  into  two  phases,  one 
using  as  problem  sources  small  areas  and  points  occupying  a  single  map  area, 
(Experiment  1)  and  one  using  large  areas,  some  of  which  were  covered  by 
more  than  one  map  area  (Experiment  2) .  The  latter  problems  included  those 
which  required  several  grid  areas  for  solution. 

For  using  the  textual  index  in  Experiment  2,  a  special  bookkeeping 
procedure  and  two  special  forms  were  developed.  A  recording  form  was 
developed  for  listing  the  grid  areas  for  each  problem  (figure  C-3) •  The 
prime  area  designators  (FI,  A3,  etc.)  were  listed  in  the  top  row  while 
the  minor  area  designators  (a,  c,  etc.)  were  listed  in  the  second  row. 

As  the  index  records  were  searched,  for  each  record  that  had  one  or  more 
hits  (covered  one  or  more  grid  areas)  the  accession  number  was  recorded 
and  an  X  placed  in  all  appropriate  columns.  Given  a  required  number  of 
records  for  a  solution  (l,  2,  or  more),  the  subject  could  reach  a  solu¬ 
tion  by  determining  if  all  the  grid  anas  were  covered  by  an  X  using  the 
correct  number  of  index  records.  Figure  C-4  illustrates  a  solution  for 
a  problem  requiring  2  index  records,  and  occupying  two  map  areas.  When 
the  problem  occupied  more  than  a  single  map  base,  the  grid  requirements 
were  indicated  on  the  same  recording  form  but  were  separated  by  a  heavy 
vertical  line.  In  Figure  C-k,  this  line  separates  Fkd  and  Aka. 

To  simplify  the  task  of  comparing  all  the  grid  areas  required  with 
each  index  record,  a  transparent  index  record  overlay  was  developed.  It 
is  basically  a  blank  textual  index  record  (Figure  C-3)  printed  on  trans¬ 
parent  acetate.  The  subject,  using  a  grease  pencil,  indicated  each  of 
the  grid  areas  requiring  coverage  by  encircling  the  appropriate  box  or 
putting  an  X  in  it.  If,  as  he  went  through  an  index  record,  he  sus¬ 
pected  there  were  one  or  more  grid  areas  of  interest  upon  it,  he  placed 
the  overlay  over  the  record  and  quickly  determined  if  the  index  record 
provided  any  coverage. 

ex?e?lme:jtal  problem  areas. 


Eight  problems  were  developed  for  each  experiment. 

Experiment  1.  Problems  1  through  3  were  points  (two  intersecting 
lines)  or  small  areas  marked  in  heavy  black  ink  on  a  transparent  acetate 
overlay  attached  to  the  prime  map  area,  Xaegumg&ng .  Figure  0-5  shows 
the  approximate  location  and  sice  of  each  of  the  o  problems.  For 
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testing,  each  problem  was  on  an  overlay  attached  to  a  separate  map. 
Problems  1,  2,  5>  and  7  were  small  areas  (smaller  than  the  smallest  grid 
area  in  size),  while  problems  3,  4,  6,  and  8  were  point  locations.  To 
provide  for  training  of  the  subjects,  four  additional  problems  were  pre¬ 
pared;  these  problems  included  2  small  area  problems  and  2  point  problems. 
In  Figure  C-5,  these  training  problems  are  designated  T-l,  T-2,  T-3,  and 
T-4.  The  training  problems  were  for  familiarizing  subjects  with  both 
types  of  index. 

For  the  graphic  index,  the  problems  were  presented  to  the  subject 
without  a  grid.  For  the  textual  problems,  the  transparent  grid  overlay 
was  oriented  correctly  to  the  map  area. 

All  problems  in  Experiment  1  required  a  single  accession  number  for 
solution;  however,  seme  problems  had  more  than  one  solution.  This  was 
equated  for  each  problem  between  the  indexes.  Thus  Problems  1,  3,  4,  5, 

6,  and  7  had  one  solution  when  using  either  index,  while  Problems  2  and  Q 
had  two  solutions  when  using  either  index. 
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Figure  C-5.  Problems  for  ’  xperiment  1  (Grid  over  mop  area) 


Experiment  2.  A  set  of  eight  problem  areas  vas  developed,  each 
greater  in  size  than  a  single  grid  area.  Four  problems  (?,  6,  and  8) 

were  located  so  as  to  fall  into  both  the  primary  map  area  (.Naegumgang) 
and  the  secondary  map  area  (Kosong).  As  in  Experiment  1,  each  problem 
area  vas  outlined  in  heavy  black  ink  on  a  transparent  acetate  overlay 
on  one  or  two  maps.  Figure  C-6  shows  the  approximate  location  and 
boundaries  of  each  of  these  problems.  In  addition,  two  training  problems 
were  prepared  (T-l  and  T-2  in  Figure  C-6) .  As  in  Experiment  1,  these 
training  problems  were  used  for  training  on  both  index  methods.  In 
Experiment  2,  in  order  to  equate  the  difficulty  of  a  problem  for  each 
type  of  index,  the  textual  records  were  adjusted  so  that  for  a  given 
problem,  say  Problem  4,  the  same  number  of  graphic  and  textual  records 
was  required  for  obtaining  a  solution.  There  was  but  one  unique  solu¬ 
tion  for  each  problem.  Two  accession  numbers  were  necessary  for  problems 
1,  2,  4,  5,  6,  and  7  and  three  accession  numbers  for  problems  3  and  8. 


SUBJECT  AM  EXPERIMENTER  RECORDING  FORMS. 

For  each  experiment,  the  subject  vas  given  a  copy  of  a  special  form 
for  recording  the  accession  number  (or  numbers)  to  each  problem  (see 
Figure  C-7) •  As  he  solved  each  problem,  he  would  fill  in  the  accession 
number ( s)  . 

The  experimenter  used  the  form  shown  in  Figure  C-8  to  record  the 
time  in  seconds  required  for  correctly  solving  each  problem  and  to 
record  other  data --.name  of  subject,  date,  any  errors,  etc.  The  experi¬ 
menter  recorded  the  obtained  accession  number  to  the  right  of  the  time 
data  when  more  than  one  solution  vas  possible. 


ANSWER  LUTS. 

A  master  scoring  sheet  was  prepared  which  listed  all  the  correct 
solutions  for  each  training  and  testing  problem  for  both  the  graphic 
and  textual  index  methods.  In  addition,  for  the  textual  index  solutions 
in  Experiment  2,  a  list  of  all  index  record  accession  numbers  that  con¬ 
tributed  to  a  solution  were  listed  in  numerical  sequence  under  al 1  the 
required  grid  areas.  This  latter  list  enabled  the  experimenter  to  moni¬ 
tor  each  subject's  progress  in  using  the  textual  index. 


Figure  C-6.  Problems  for  Experiment  2  (Grid  over  each  map  area). 
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